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ABSTRACT 

Assessments have risen in prominence in colleges of business, in response to 
requirements of accrediting agencies. Among the forms of assessment are embedded exams 
within courses, often in the form of multiple-choice tests near the end of the semester. These tests 
can be stand-alone comprehensive exercises, or comprise a small portion of a larger exam. 
Performing well on exit exams such as the ETS is highly dependent on how well students did in 
their foundational courses. This study examines outcomes assessment scores in 11 sections of the 
Principles of Marketing, Management, and Business Law courses at a southwestern university. 
These courses were taught by a variety of full-time and part-time faculty. Models were 
constructed based on demographic variables as well as native ability, effort, and type of 
instructor. The results indicate that part-time instructors in these courses had exam scores 15% 
lower than those of students in courses taught by full-time professors. Furthermore, it was 
observed that students with adequate native ability (as measured by their ACT score) and effort 
(as measured by their cumulative GPA), could overcome the effects of poor instruction by part- 
time faculty. 
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INTRODUCTION 

Increasingly, academic programs look to assessment as an important vehicle in 
determining the overall veracity of a program. It is a means by which the deliverer (instructor) 
of an academic program can determine if the goals and objectives of the program have been 
achieved by the recipients (students). Institutional effectiveness is concerned with the extent to 
which intended outcomes are being achieved (Black & Duhon, 2003). There are two (2) 
fundamental schemas of assessment delivery. The first employs an instrument developed 
externally and is standardized across a multitude of dimensions, like the Educational Testing 
Service's (ETS) exam in business. The second approach employs an embedded, internally 
developed, instrument that explicitly measures the specific outcomes associated with a program 
or course. 

While much research has been done on the ETS and its predictors of student 
performance, little has been done to assess the impact of the type of instructor. Furthermore, 
most assessment-related research has focused on the capstone course in the program, rather than 
the earlier foundational courses. The purpose of this paper is to examine the impact of using full¬ 
time and part-time instructors on student performance on assessment exams in three foundational 
courses (Marketing Management, and Business Law). Assessments were administered to 11 
sections of students of these three courses at a College of Business at a public university located 
in the Southwestern part of the United States. The institution is mid-sized with a total enrollment 
of approximately 7,500 total students, 1,000 undergraduate business students, 350 graduate 
business students. 

This manuscript is organized as follows: First, a literature review is provided. The 
second section of the manuscript describes the data method, model, and results. This includes 
detailed examination and analysis of the differences observed. The final section offers 
conclusions, implications, and directions for future research. 

LITERATURE REVIEW 

Research abounds on the subject of program assessment, but not on type of instructor. 
Assessment is a "systematic collection, review, and use of information about educational 
programs undertaken for the purpose of improving student learning and development" ( Palomba 
& Banta, 1999). Collegiate business programs are increasingly tasked with the need for ongoing 
assessment of student performance in their programs (Adams, et al, 2000; Bagamery, et al, 2005; 
Martell & Calderon, 2005; Terry, et al, 2008; Trapnell, 2005). Increasingly, since the mid-1980s, 
there has been a shift towards student-centered and learning-oriented assessments and 
accreditation (Lubinescu, et al, 2001). In fact, the AACSB imposes standards for program 
learning goals upon collegiate business programs aspiring to attain or maintain AACSB 
accreditation. These programs utilize direct measures in order to demonstrate student 
achievement of the stipulated goals (Martell, 2007; Pringle & Michel, 2007). 

As assessment increases to build momentum, it is important to identify the internal and 
external audiences who will utilize the results in shaping and refining the assessment process. A 
comprehensive assessment process provides an institution with information that can be both 
shared and utilized to satisfy the needs of internal and external constituents. The internal 
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audience (faculty, students, assessment committees, administrators and alumni) benefits by 
helping to define successful ongoing programs, implementing similar programs, and for 
improving less successful programs. Externally, the assessment data are used to demonstrate to 
accreditation organizations, government officials, government boards and other constituents the 
institution's effectiveness and accountability (Aloi, et al, 2003). 

There are two principle types of assessment tests, standardized and local. Of the two, 
local tests require more faculty effort and other resources for test development, scoring, 
reporting, and improving. However, the advantage to the local instrument is that it can be 
tailored to a specific course or program so that the actual scores more accurately reflect the 
extent to which specific learning objectives are being met ( Black & Duhon, 2003), along with 
impact of local and specific influences or drivers. 

The Educational Testing Service's (ETS) exam in business has become, to many, the de 
facto standard of standardized assessment instruments in collegiate business programs. The 
literature reveals an almost universal agreement as to the principle variables examined as 
predictors of student performance on the ETS exam. These variables include: grade point 
average (GPA), standardized test scores (ACT/SAT) and gender. In addition to these variables, 
Mirchandani, et al, (2001) include transfer GPA and student grades in quantitative courses. 

It is possible to extrapolate and utilize the same variables when examining the results on 
a local instrument. Terry, et al, (2008) developed a model based upon a production view of 
student learning to examine the determinants of performance on the business major field 
achievement ETS exam. Their model controlled for grade point average (GPA), standardized 
test scores (SAT/ACT), junior college transfer students, gender. Their findings were consistent 
with much of the previous research in this area, that academic ability as measured by grade point 
average (GPA) and scores on standardized tests (SAT/ACT) are the primary determinants of 
student performance on the ETS exam. 

Black & Duhon's (2003) study of ETS scores conducted during three (3) semesters in 
1996-1997 included an examination of an incentive as a driver in student performance. In that 
study, students scoring at the national 50th percentile, or better, were given an extra-point bonus, 
which was used in the calculation of the student's final course grade. The exclusion of this, or 
some other, form of incentive is not used; some students may not take the test seriously and by 
extension, the results may be misleading (Allen & Bycio, 1997). Terry (2007) included the 
impact of course formats, traditional campus courses, online courses and the newer hybrid 
courses on ETS scores. 

While studies on ETS outcomes are abundant, there is a paucity of research examining 
the effects of the type of instructors used in business courses. Furthermore, the majority of 
assessment-related research focuses exclusively on outcomes and student traits, and not on the 
employment status of their instructors. Part-time instructors are frequently used to fill gaps in the 
delivery capacity of institutions. While these instructors are normally professionally qualified in 
the discipline themselves, they often lack the training and pedagogical background found in a 
full-time faculty member. Depending on the institution, full-time faculty members may teach up 
to three or four courses per term, while part-time faculty may teach only one or two (while often 
balancing other employment obligations). 

Although prior research has shown that a student’s ETS score can be predicted in large 
part by traits such as ACT/SAT score, GPA, gender, and the like, what is lacking is understanding 
the drivers in a student’s performance in foundational course assessments. 
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METHODOLOGY, MODELS AND RESULTS 

Embedded multiple-choice assessment exams were administered in the Spring 2008 
semester at a four-year public university with 7500 students in the southern US. In all cases the 
respective exams included 20 questions, each worth 5 points. Courses included in this study and 
the mean scores are: Principles of Marketing (3 section; mean = 73.38), Principles of 
Management (4 sections; mean = 66.14), and Business Law (4 sections; mean = 74.61). All three 
courses are at the Junior level, and are offered within the same academic department. 

Demographic data were collected from the Registrar’s office, including gender, 
nationality, transfer status, GPA (effort), and ACT (native ability). The first three variables were 
coded as (0) or (1) binary variables, while GPA ranged from 0.00 to 4.00, and act ranged from 1 
to 36. ACT scores were only available for a subset of the overall sample, because this test score 
is not required for matriculating transfer at this institution. A final (0) or (1) binary variable was 
created signifying whether the course instructor was full-time or part-time. 

The first round of modeling is the Full Model, which included 350 course-students and 
their exam scores. Data from all 11 sections were aggregated without regard to course or section. 
There were four course sections taught by part-time instructors (112 course-students), and seven 
courses taught by full-time professors (238 course-students). This group contained students both 
with and without an ACT score; the ACT score was not factored into this first model. The overall 
mean assessment score across the entire sample was 71.0. 

A linear regression model was then calculated with Comp (score on the comprehensive 
assessment exam) serving as the dependent variable, which is a function of the following: 

Comp = f(Demographics + Effort + Professor) 

Specifically, the model is as follows: 

COMP(i) = B(0) + B(l)GENDER(i) + B (2)TRANSFER(i) + B(3)NATIONALITY (i) + 
B(4)GPA(i) + B(5)PROF(i) + e(i) 

Results from this model appear in Table 1. With an R 2 of 0.26, the GPA and PROF 
variables were significant at the p = 0.05 level. Thus, the student’s cumulative GPA and type of 
instructor serve as strong predictors of student scores on their assessment exams. 

A t-test for independent means was then calculated using COMP as the dependent 
variable and PROF as the predictor. The mean score for students taught by a part-time instructor 
was 60.9, while the mean score for those taught by a full-time instructor was 75.9 (t=-9.168; p = 
0 . 000 ). 

A second model was then built among the subset of students with a reported ACT score. 
The average test score among these 153 students was 69.87. This model was conceptualized and 
specified as: 

Comp = f(Demographics + Effort + Native Ability + Professor) 

COMP(i) = B(0) + B(l)GENDER(i) + B (2)TRANSFER(i) + B(3)NATIONALITY(i) + 
B(4)GPA(i) + B(5)ACT(i) + B(6)PROF(i) + e(i) 
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Results from this model appear in Table 2. With an R 2 of 0.305, the ACT and PROF 
variables were significant at the p = 0.05 level, while GPA was significant at p = 0.06. It can be 
concluded that, while PROF is a major predictor of student outcomes, both ACT and GPA play 
an important role in student scores, with ACT being the strongest predictor. More importantly, it 
suggests that native ability (ACT) and effort (GPA) are sufficient to overcome a weak part-time 
instructor’s teaching performance. 

A t-test for independent means was calculated for the ACT subset using COMP as the 
dependent variable and PROF as the predictor. The results were similar to those for the full data 
set; the mean score for students taught by a part-time instructor was 58.6, while the mean score 
for those taught by a full-time instructor was 74.9 (t=-6.563; p = 0.000). 

Next, correlations were calculated among the five independent variables as a test for 
multicollinearity. The highest correlation was .295, indicating there was no multicollinearity 
among these variables. 

Correlations were then run among two independent variables (GPA and ACT) with the 
dependent COMP variable, first for the ACT subset (Table 4), and then for a further refined 
subset of ACT students taught by part-time instructors (Table 5). In the first case, both GPA and 
ACT correlated significantly with COMP, verifying the conclusion of the regression equation. In 
the second case, ACT correlated with COMP at the p=0.01 level, with GPA correlating at p=0.07. 
These results reinforce the conclusion above the native ability (ACT) and effort (GPA) are 
sufficient for overcoming a poor classroom experience, with native ability being the foremost 
predictor. 

CONCLUSIONS AND FUTURE RESEARCH 

This study is limited in that it was conducted in only one semester, at one institution, and 
among only a portion of course offerings in the College of Business. Thus, findings may not be 
generalizable across time, place, and subject. Still, the findings suggest that Colleges of Business 
must exercise prudence in the hiring of part-time faculty who probably have other means of 
employment, and thus may treat their course(s) as of secondary importance. This is not to say 
that all part-time faculty are of inferior quality or that full-time faculty are of superior quality; 
rather, it signifies that caution must be used in hiring decisions. Furthermore, part-time faculty by 
definition do not necessarily have the mission of the college or university at heart; it is likely 
they are gainfully employed elsewhere, and thus teaching one or two courses as a source of 
supplemental income. 

Future research should examine a broader spectrum of course offerings in the College of 
Business, and, if possible, beyond principles-level courses. It would also be of great interest to 
track the 350 students in this study as they complete their education to see how they score on 
assessments in the capstone senior-level course. It is possible that a poor foundation will lead to 
poor scores later in their academic careers. One implication of these results is that part-time 
faculty should be used for elective courses rather than foundational courses, wherever possible. 

Finally, it should be noted that, while students with high ability and effort in this study 
managed to overcome the effects of a poor instructor, care should be taken at administrative 
levels to not expect learning to occur in spite of who teaches the course. 
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Table 1: The Full Model 


Coefficients(a) 


Mode 

1 

Unstandardized 

Coefficients 

Standardized 

Coefficients 

t 

Sig- 

B 

Std. 

Error 

Beta 

B 

Std. 

Error 

1 (Constant 

) 

42.811 

3.855 


11.106 

.000 

Gender 

-.496 

1.491 

-.016 

-.332 

.740 

Transfer 

2.466 

2.124 

.056 

1.161 

.246 

GPA 

5.713 

1.079 

.248 

5.293 

.000 

Nat 

1.574 

3.613 

.021 

.436 

.663 

Prof 

13.962 

1.609 

.410 

8.677 

.000 


(a) Dependent Variable: Comp 


Table 2: The Partial Model 


Coefficients(a) 


Mode 

1 

Unstandardized 

Coefficients 

Standardized 

Coefficients 

t 

Sig- 

B 

Std. 

Error 

Beta 

B 

Std. 

Error 

1 (Constant 

) 

23.876 

10.366 


2.303 

.023 

Gender 

-1.461 

2.301 

-.045 

-.635 

.527 

Transfer 

2.978 

2.902 

.074 

1.026 

.307 

GPA 

3.893 

2.039 

.144 

1.909 

.058 

Nat 

4.514 

6.070 

.055 

.744 

.458 

ACT 

.996 

.338 

.213 

2.943 

.004 

Prof 

13.904 

2.502 

.403 

5.557 

.000 


(a) Dependent Variable: Comp 
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Table 3: Correlations Among Partial Model Independent Variables 

Correlations 



Gender 

Transfer 

GPA 

Nat 

ACT 

Prof 

Gender 

Pearson 

Correlation 

1 

.042 

-.162(*) 

-.167(*) 

-.129 

.001 


Sig. (2-tailed) 


.608 

.046 

.039 

.112 

.991 


N 

153 

153 

153 

153 

153 

153 

Transfe 

Pearson 

.042 

1 

.118 

- 

.154 

-.008 

r 

Correlation 

,239(**) 


Sig. (2-tailed) 

.608 


.147 

.003 

.057 

.925 


N 

153 

153 

153 

153 

153 

153 

GPA 

Pearson 

Correlation 

-,162(*) 

.118 

1 

.149 

.183(0 

.295(**) 


Sig. (2-tailed) 

.046 

.147 


.067 

.023 

.000 


N 

153 

153 

153 

153 

153 

153 

Nat 

Pearson 

Correlation 

-.167(*) 

,239(**) 

.149 

1 

-.150 

.062 


Sig. (2-tailed) 

.039 

.003 

.067 


.064 

.450 


N 

153 

153 

153 

153 

153 

153 

ACT 

Pearson 

Correlation 

-.129 

.154 

.183(0 

-.150 

1 

.105 


Sig. (2-tailed) 

.112 

.057 

.023 

.064 


.197 


N 

153 

153 

153 

153 

153 

153 

Prof 

Pearson 

Correlation 

.001 

-.008 

.295(**) 

.062 

.105 

1 


Sig. (2-tailed) 

.991 

.925 

.000 

.450 

.197 



N 

153 

153 

153 

153 

153 

153 


* Correlation is significant at the 0.05 level (2-tailed). 
** Correlation is significant at the 0.01 level (2-tailed). 
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Table 4: Correlations Among Partial Model Selected Variables 



GPA 

ACT 

Comp 

GPA 

Pearson 

Correlation 

1 

.183(0 

.312(*0 


Sig. (2-tailed) 


.023 

.000 


N 

153 

153 

153 

ACT 

Pearson 

Correlation 

.183(*) 

1 

.280(*O 


Sig. (2-tailed) 

.023 


.000 


N 

153 

153 

153 

Comp 

Pearson 

Correlation 

.312(**) 

.280(*O 

1 


Sig. (2-tailed) 

.000 

.000 



N 

153 

153 

153 


Correlation is significant at the 0.05 


evel (2-tailed). 




Correlation is significant at the 0.01 level (2-tailed). 


Table 5: Correlations Among Partial Model Selected Variables: 
The Case of the Part-Time Instructor 


Correlations 



GPA 

ACT 

Comp 

GPA 

Pearson 

Correlation 

1 

.162 

.267 


Sig. (2-tailed) 


.278 

.070 


N 

47 

47 

47 

ACT 

Pearson 

Correlation 

.162 

1 

.379(**) 


Sig. (2-tailed) 

.278 


.009 


N 

47 

47 

47 

Comp 

Pearson 

Correlation 

.267 

.379(*0 

1 


Sig. (2-tailed) 

.070 

.009 



N 

47 

47 

47 


** Correlation is significant at the 0.01 level (2-tailed). 
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